{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {
    "id": "0"
   },
   "source": [
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/geoai/blob/main/docs/workshops/CANVAS_2025.ipynb)\n",
    "\n",
    "**Open Source Pipeline to Integrate Drone and Satellite Geospatial Data Products for Agricultural Applications**\n",
    "\n",
    "This notebook is designed for workshop presented at the [CANAVS 2025 Conference](https://www.sciencesocieties.org/canvas?q=canvas/) on November 10, 2025.\n",
    "\n",
    "-   Registration: <https://scisoc.confex.com/scisoc/2025am/meetingapp.cgi/Session/27796>\n",
    "-   Notebook: <https://opengeoai.org/workshops/CANVAS_2025>\n",
    "-   Leafmap: <https://leafmap.org>\n",
    "-   Samgeo: <https://samgeo.gishub.org>\n",
    "-   GeoAI: <https://opengeoai.org>\n",
    "-   Data to Science (D2S): <https://ps2.d2s.org>\n",
    "-   D2S Python API: <https://py.d2s.org>\n",
    "\n",
    "## Introduction\n",
    "\n",
    "Recent advances in drone technology have revolutionized the remote sensing community by providing means to collect fine spatial and high temporal resolutions at affordable costs. As people are gaining access to increasingly larger volumes of drone and satellite geospatial data products, there is a growing need to extract relevant information from the vast amount of freely available geospatial data. However, the lack of specialized software packages tailored for processing such data makes it challenging to develop transdisciplinary research collaboration around them. This workshop aims to bridge the gap between big geospatial data and research scientists by providing training on an open-source online platform for managing big drone data known as Data to Science.\n",
    "\n",
    "## Agenda\n",
    "\n",
    "The main topics to be covered in this workshop include:\n",
    "\n",
    "* Create interactive maps using leafmap\n",
    "* Visualize drone imagery from D2S\n",
    "* Automated segmentation of drone imagery using SAMGeo\n",
    "* Training a U-Net model for tree segmentation\n",
    "\n",
    "## Environment setup\n",
    "\n",
    "[![image](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/opengeos/geoai/blob/main/docs/workshops/CANVAS_2025.ipynb)\n",
    "\n",
    "\n",
    "### Change runtime type to GPU\n",
    "\n",
    "To speed up the processing, you can change the Colab runtime type to GPU. Go to the \"Runtime\" menu, select \"Change runtime type\", and choose \"T4 GPU\" from the \"Hardware accelerator\" dropdown menu.\n",
    "\n",
    "![image](https://github.com/user-attachments/assets/e92d2a19-0555-456d-b4be-36680c0af09f)\n",
    "\n",
    "\n",
    "### Install packages\n",
    "\n",
    "Uncomment the following code to install the required packages."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {
    "id": "1"
   },
   "outputs": [],
   "source": [
    "%pip install -U \"leafmap[raster]\" \"segment-geospatial[samgeo2]\" geoai-py"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {
    "id": "2"
   },
   "source": [
    "### Import libraries\n",
    "\n",
    "Import the necessary libraries for this workshop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {
    "id": "3"
   },
   "outputs": [],
   "source": [
    "import leafmap"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {
    "id": "4"
   },
   "source": [
    "## Creating interactive maps\n",
    "\n",
    "Let's create an interactive map using the `ipyleaflet` plotting backend. The [`leafmap.Map`](https://leafmap.org/leafmap/#leafmap.leafmap.m) class inherits the [`ipyleaflet.Map`](https://ipyleaflet.readthedocs.io/en/latest/map_and_basemaps/map.html) class. Therefore, you can use the same syntax to create an interactive map as you would with `ipyleaflet.Map`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {
    "id": "5"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {
    "id": "6"
   },
   "source": [
    "To display it in a Jupyter notebook, simply ask for the object representation:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {
    "id": "7"
   },
   "outputs": [],
   "source": [
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {
    "id": "8"
   },
   "source": [
    "To customize the map, you can specify various keyword arguments, such as `center` ([lat, lon]), `zoom`, `width`, and `height`. The default `width` is `100%`, which takes up the entire cell width of the Jupyter notebook. The `height` argument accepts a number or a string. If a number is provided, it represents the height of the map in pixels. If a string is provided, the string must be in the format of a number followed by `px`, e.g., `600px`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {
    "id": "9"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(center=[40, -100], zoom=4, height=\"600px\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {
    "id": "10"
   },
   "source": [
    "### Adding basemaps\n",
    "\n",
    "There are several ways to add basemaps to a map. You can specify the basemap to use in the `basemap` keyword argument when creating the map. Alternatively, you can add basemap layers to the map using the `add_basemap` method. leafmap has hundreds of built-in basemaps available that can be easily added to the map with only one line of code.\n",
    "\n",
    "Create a map by specifying the basemap to use as follows. For example, the `Esri.WorldImagery` basemap represents the Esri world imagery basemap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {
    "id": "11"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map(basemap=\"Esri.WorldImagery\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {
    "id": "12"
   },
   "source": [
    "You can add as many basemaps as you like to the map. For example, the following code adds the `OpenTopoMap` basemap to the map above:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {
    "id": "13"
   },
   "outputs": [],
   "source": [
    "m.add_basemap(\"OpenTopoMap\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {
    "id": "14"
   },
   "source": [
    "You can also add an XYZ tile layer to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {
    "id": "15"
   },
   "outputs": [],
   "source": [
    "basemap_url = \"https://mt1.google.com/vt/lyrs=y&x={x}&y={y}&z={z}\"\n",
    "m.add_tile_layer(basemap_url, name=\"Hybrid\", attribution=\"Google\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {
    "id": "16"
   },
   "source": [
    "You can also change basemaps interactively using the basemap GUI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {
    "id": "17"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_basemap_gui()\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {
    "id": "18"
   },
   "source": [
    "## Visualizing Drone Imagery from D2S\n",
    "\n",
    "The Data to Science (D2S) platform (https://ps2.d2s.org) hosts a large collection of drone imagery that can be accessed through the D2S API (https://py.d2s.org). To visualize drone imagery from D2S, you need to [sign up](https://ps2.d2s.org/auth/register) for a free account on the D2S platform and obtain an API key."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "19",
   "metadata": {
    "id": "19"
   },
   "source": [
    "### Authenticate with D2S Platform\n",
    "\n",
    "Connect to your D2S workspace using your credentials. The workshop provides a demo account for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20",
   "metadata": {
    "id": "20"
   },
   "outputs": [],
   "source": [
    "import os\n",
    "from datetime import date\n",
    "from d2spy.workspace import Workspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {
    "id": "21"
   },
   "outputs": [],
   "source": [
    "if os.environ.get(\"D2S_EMAIL\") is None:\n",
    "    os.environ[\"D2S_EMAIL\"] = \"qwu18@utk.edu\"  # Replace with your email address"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "22",
   "metadata": {
    "id": "22"
   },
   "outputs": [],
   "source": [
    "# Connect to D2S platform and authenticate\n",
    "d2s_url = \"https://ps2.d2s.org\"\n",
    "workspace = Workspace.connect(d2s_url)  # This will prompt for password\n",
    "api_key = workspace.api_key  # Get API key for data access"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {
    "id": "23"
   },
   "outputs": [],
   "source": [
    "os.environ[\"D2S_API_KEY\"] = api_key\n",
    "os.environ[\"TITILER_ENDPOINT\"] = \"https://tt.d2s.org\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24",
   "metadata": {
    "id": "24"
   },
   "source": [
    "### Choose a project to work with\n",
    "\n",
    "The Workspace `get_projects` method will retrieve a collection of the projects your account can currently access on the D2S instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {
    "id": "25"
   },
   "outputs": [],
   "source": [
    "# Get list of all your projects\n",
    "projects = workspace.get_projects()\n",
    "for project in projects:\n",
    "    print(project)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26",
   "metadata": {
    "id": "26"
   },
   "source": [
    "The `projects` variable is a `ProjectCollection`. The collection can be filtered by either the project descriptions or titles using the methods `filter_by_title` or `filter_by_name`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27",
   "metadata": {
    "id": "27"
   },
   "outputs": [],
   "source": [
    "# Example of creating new collection of only projects with the keyword \"Citrus Orchard\" in the title\n",
    "filtered_projects = projects.filter_by_title(\"Citrus Orchard\")\n",
    "print(filtered_projects)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28",
   "metadata": {
    "id": "28"
   },
   "source": [
    "Now you can choose a specific project to work with. In this case, the filtered projects returned only one project, so we will use that project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {
    "id": "29"
   },
   "outputs": [],
   "source": [
    "project = filtered_projects[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30",
   "metadata": {
    "id": "30"
   },
   "source": [
    "### Get the project boundary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31",
   "metadata": {
    "id": "31"
   },
   "source": [
    "`get_project_boundary` method of the `Project` class will retrieve a GeoJSON object of the project boundary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32",
   "metadata": {
    "id": "32"
   },
   "outputs": [],
   "source": [
    "# Get project boundary as Python dictionary in GeoJSON structure\n",
    "project_boundary = project.get_project_boundary()\n",
    "project_boundary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33",
   "metadata": {
    "id": "33"
   },
   "source": [
    "### Get project flights\n",
    "\n",
    "The `Project` `get_flights` method will retrieve a list of flights associated with the project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34",
   "metadata": {
    "id": "34"
   },
   "outputs": [],
   "source": [
    "# Get list of all flights for a project\n",
    "flights = project.get_flights()\n",
    "# Print first flight object (if one exists)\n",
    "for flight in flights:\n",
    "    print(flight)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35",
   "metadata": {
    "id": "35"
   },
   "source": [
    "### Filter flights by date\n",
    "\n",
    "The `flights` variable is a `FlightCollection`. The collection can be filtered by the acquisition date using the method `filter_by_date`. This method will return all flights with an acquisition date between the provided start and end dates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36",
   "metadata": {
    "id": "36"
   },
   "outputs": [],
   "source": [
    "# Example of creating new collection of only flights from June 2022\n",
    "filtered_flights = flights.filter_by_date(\n",
    "    start_date=date(2022, 6, 1), end_date=date(2022, 7, 1)\n",
    ")\n",
    "for flight in filtered_flights:\n",
    "    print(flight)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "37",
   "metadata": {
    "id": "37"
   },
   "source": [
    "Now, we can choose a flight from the filtered flight. Let's choose the flight on June 9, 2022."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "38",
   "metadata": {
    "id": "38"
   },
   "outputs": [],
   "source": [
    "flight = filtered_flights[0]\n",
    "flight"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39",
   "metadata": {
    "id": "39"
   },
   "source": [
    "### Get data products\n",
    "\n",
    "The Flight `get_data_products` method will retrieve a list of data products associated with the flight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40",
   "metadata": {
    "id": "40"
   },
   "outputs": [],
   "source": [
    "# Get list of data products from a flight\n",
    "data_products = flight.get_data_products()\n",
    "\n",
    "for data_product in data_products:\n",
    "    print(data_product)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41",
   "metadata": {
    "id": "41"
   },
   "source": [
    "The `data_products` variable is a `DataProductCollection`. The collection can be filtered by data type using the method `filter_by_data_type`. This method will return all data products that match the requested data type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42",
   "metadata": {
    "id": "42"
   },
   "outputs": [],
   "source": [
    "# Example of creating new collection of data products with the \"ortho\" data type\n",
    "ortho_data_products = data_products.filter_by_data_type(\"ortho\")\n",
    "print(ortho_data_products)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43",
   "metadata": {
    "id": "43"
   },
   "source": [
    "### Visualize ortho imagery\n",
    "\n",
    "Now we can grab the ortho URL to display it using leafmap."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "44",
   "metadata": {
    "id": "44"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_basemap(\"HYBRID\", show=False)\n",
    "ortho_data = ortho_data_products[0]\n",
    "ortho_url_202206 = ortho_data.url\n",
    "ortho_url_202206 = leafmap.d2s_tile(ortho_url_202206)\n",
    "m.add_cog_layer(ortho_url_202206, name=\"Ortho Imagery 202206\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45",
   "metadata": {
    "id": "45"
   },
   "source": [
    "### Visualize DSM\n",
    "\n",
    "Similarly, you can visualize the Digital Surface Model (DSM) from D2S using the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46",
   "metadata": {
    "id": "46"
   },
   "outputs": [],
   "source": [
    "# Example of creating new collection of data products with the \"dsm\" data type\n",
    "dsm_data_products = data_products.filter_by_data_type(\"dsm\")\n",
    "print(dsm_data_products)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47",
   "metadata": {
    "id": "47"
   },
   "outputs": [],
   "source": [
    "dsm_data = dsm_data_products[0]\n",
    "dsm_url_202206 = dsm_data.url\n",
    "dsm_url_202206 = leafmap.d2s_tile(dsm_url_202206)\n",
    "m.add_cog_layer(dsm_url_202206, colormap_name=\"terrain\", name=\"DSM 202206\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "48",
   "metadata": {
    "id": "48"
   },
   "outputs": [],
   "source": [
    "leafmap.cog_stats(dsm_url_202206)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49",
   "metadata": {
    "id": "49"
   },
   "source": [
    "Add a colorbar to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50",
   "metadata": {
    "id": "50"
   },
   "outputs": [],
   "source": [
    "m.add_colormap(cmap=\"terrain\", vmin=3, vmax=33, label=\"Elevation (m)\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51",
   "metadata": {
    "id": "51"
   },
   "source": [
    "### Visualize CHM\n",
    "\n",
    "Similarly, you can visualize the Canopy Height Model (CHM) from D2S using the code below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "52",
   "metadata": {
    "id": "52"
   },
   "outputs": [],
   "source": [
    "# Example of creating new collection of data products with the \"chm\" data type\n",
    "chm_data_products = data_products.filter_by_data_type(\"chm\")\n",
    "print(chm_data_products)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53",
   "metadata": {
    "id": "53"
   },
   "outputs": [],
   "source": [
    "chm_data = chm_data_products[0]\n",
    "chm_url_202206 = chm_data.url\n",
    "chm_url_202206 = leafmap.d2s_tile(chm_url_202206)\n",
    "m.add_cog_layer(chm_url_202206, colormap_name=\"jet\", name=\"CHM 202206\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54",
   "metadata": {
    "id": "54"
   },
   "outputs": [],
   "source": [
    "leafmap.cog_stats(chm_url_202206)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55",
   "metadata": {
    "id": "55"
   },
   "outputs": [],
   "source": [
    "m.add_colormap(cmap=\"jet\", vmin=0, vmax=13, label=\"Elevation (m)\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56",
   "metadata": {
    "id": "56"
   },
   "source": [
    "Add the project boundary to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57",
   "metadata": {
    "id": "57"
   },
   "outputs": [],
   "source": [
    "m.add_geojson(project_boundary, layer_name=\"Project Boundary\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58",
   "metadata": {
    "id": "58"
   },
   "source": [
    "Add tree boundaries to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59",
   "metadata": {
    "id": "59"
   },
   "outputs": [],
   "source": [
    "map_layers = project.get_map_layers()\n",
    "tree_boundaries = map_layers[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60",
   "metadata": {
    "id": "ralhPvgwN2Tv"
   },
   "outputs": [],
   "source": [
    "gdf = leafmap.geojson_to_gdf(tree_boundaries)\n",
    "gdf.to_file(\"tree_boundaries.geojson\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "61",
   "metadata": {
    "id": "60"
   },
   "outputs": [],
   "source": [
    "m.add_geojson(tree_boundaries, layer_name=\"Tree Boundaries\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62",
   "metadata": {
    "id": "61"
   },
   "source": [
    "### Get another flight\n",
    "\n",
    "Retrieve the Ortho data product for the December 2022 flight."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63",
   "metadata": {
    "id": "62"
   },
   "outputs": [],
   "source": [
    "filtered_flights = flights.filter_by_date(\n",
    "    start_date=date(2022, 12, 1), end_date=date(2022, 12, 31)\n",
    ")\n",
    "for flight in filtered_flights:\n",
    "    print(flight)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "64",
   "metadata": {
    "id": "63"
   },
   "outputs": [],
   "source": [
    "flight_202212 = filtered_flights[0]\n",
    "data_products = flight_202212.get_data_products()\n",
    "ortho_data_products = data_products.filter_by_data_type(\"ortho\")\n",
    "ortho_data = ortho_data_products[0]\n",
    "ortho_url_202212 = ortho_data.url\n",
    "ortho_url_202212 = leafmap.d2s_tile(ortho_url_202212)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65",
   "metadata": {
    "id": "64"
   },
   "source": [
    "### Compare two ortho images\n",
    "\n",
    "Create a split map for comparing the June 2022 and December 2022 ortho images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66",
   "metadata": {
    "id": "65"
   },
   "outputs": [],
   "source": [
    "from ipyleaflet import TileLayer\n",
    "\n",
    "m = leafmap.Map()\n",
    "left_layer = TileLayer(\n",
    "    url=leafmap.cog_tile(ortho_url_202206), max_zoom=30, name=\"2022-06 Ortho\"\n",
    ")\n",
    "right_layer = TileLayer(\n",
    "    url=leafmap.cog_tile(ortho_url_202212), max_zoom=30, name=\"2022-12 Ortho\"\n",
    ")\n",
    "m.split_map(left_layer, right_layer, left_label=\"2022-06\", right_label=\"2022-12\")\n",
    "m.set_center(-97.955281, 26.165595, 18)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67",
   "metadata": {
    "id": "66"
   },
   "source": [
    "## Download and Prepare Data for Analysis\n",
    "\n",
    "In this section, we'll download a subset of the drone imagery from D2S and prepare it for machine learning model training and inference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "68",
   "metadata": {
    "id": "67"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_cog_layer(ortho_url_202206, name=\"Ortho Imagery 202206\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69",
   "metadata": {
    "id": "68"
   },
   "source": [
    "### Define Area of Interest (AOI)\n",
    "\n",
    "Draw an area of interest (AOI) on the map using the drawing tools, or use the default bounding box provided below. This AOI will be used to clip the imagery for analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70",
   "metadata": {
    "id": "69"
   },
   "outputs": [],
   "source": [
    "# Define bounding box for the area of interest\n",
    "# Format: [min_lon, min_lat, max_lon, max_lat]\n",
    "if m.user_roi is not None:\n",
    "    bbox = m.user_roi_bounds()  # Use drawn ROI if available\n",
    "else:\n",
    "    bbox = [-97.956252, 26.165315, -97.954992, 26.165883]  # Default ROI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71",
   "metadata": {
    "id": "70"
   },
   "outputs": [],
   "source": [
    "ortho_image_202206 = \"ortho_image_202206.tif\"\n",
    "if not os.path.exists(ortho_image_202206):\n",
    "    leafmap.download_file(ortho_url_202206, output=ortho_image_202206)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72",
   "metadata": {
    "id": "71"
   },
   "outputs": [],
   "source": [
    "ortho_image_202212 = \"ortho_image_202212.tif\"\n",
    "if not os.path.exists(ortho_image_202212):\n",
    "    leafmap.download_file(ortho_url_202212, output=ortho_image_202212)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73",
   "metadata": {
    "id": "72"
   },
   "source": [
    "Draw an area of interest (AOI) on the map. If an AOI is not provided, a default AOI will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74",
   "metadata": {
    "id": "73"
   },
   "outputs": [],
   "source": [
    "if m.user_roi is not None:\n",
    "    bbox = m.user_roi_bounds()\n",
    "else:\n",
    "    bbox = [-97.956252, 26.165315, -97.954992, 26.165883]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75",
   "metadata": {
    "id": "74"
   },
   "outputs": [],
   "source": [
    "gdf = leafmap.bbox_to_gdf(bbox)\n",
    "m.add_gdf(gdf, layer_name=\"AOI\", info_mode=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "76",
   "metadata": {
    "id": "75"
   },
   "outputs": [],
   "source": [
    "ortho_image_10cm = \"ortho_image_202206_10cm.tif\"\n",
    "chm_image = \"chm_202206.tif\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77",
   "metadata": {
    "id": "76"
   },
   "outputs": [],
   "source": [
    "clipped_ortho_data_10cm = leafmap.clip_raster(\n",
    "    ortho_image_202206,\n",
    "    geometry=bbox,\n",
    "    geom_crs=\"EPSG:4326\",\n",
    "    bands=[1, 2, 3],\n",
    "    resolution=0.1,\n",
    "    output=ortho_image_10cm,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "78",
   "metadata": {
    "id": "77"
   },
   "outputs": [],
   "source": [
    "clipped_chm_data = leafmap.clip_raster(\n",
    "    chm_url_202206,\n",
    "    geometry=bbox,\n",
    "    geom_crs=\"EPSG:4326\",\n",
    "    match_raster=clipped_ortho_data_10cm,\n",
    "    output=chm_image,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "79",
   "metadata": {
    "id": "78"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(ortho_image_10cm, layer_name=\"Ortho Image 202206\")\n",
    "m.add_raster(chm_image, colormap=\"terrain\", nodata=0, layer_name=\"CHM 202206\")\n",
    "m.add_geojson(tree_boundaries, layer_name=\"Tree Boundaries\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80",
   "metadata": {
    "id": "79"
   },
   "source": [
    "## Automated Segmentation with SAM (Segment Anything Model)\n",
    "\n",
    "SAMGeo is a Python package that applies Meta's Segment Anything Model (SAM) to geospatial data. It can automatically segment objects in drone and satellite imagery without requiring training data.\n",
    "\n",
    "### Initialize SAM2 Model\n",
    "\n",
    "First, we'll initialize the SAM2 model with parameters that control segmentation quality and detail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81",
   "metadata": {
    "id": "80"
   },
   "outputs": [],
   "source": [
    "from samgeo import SamGeo, SamGeo2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "82",
   "metadata": {
    "id": "81"
   },
   "outputs": [],
   "source": [
    "# Initialize SAM2 model for automatic segmentation\n",
    "sam2 = SamGeo2(\n",
    "    model_id=\"sam2-hiera-large\",  # Use the large hierarchical SAM2 model\n",
    "    automatic=True,  # Enable automatic mask generation mode\n",
    "    stability_score_thresh=0.9,  # Higher = fewer, more stable masks\n",
    "    stability_score_offset=0.7,  # Adjusts stability calculation\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83",
   "metadata": {
    "id": "82"
   },
   "source": [
    "### Generate Segmentation Masks\n",
    "\n",
    "Run SAM2 on the ortho imagery to automatically detect and segment all objects. This process may take a few minutes depending on image size and computational resources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84",
   "metadata": {
    "id": "83"
   },
   "outputs": [],
   "source": [
    "# Generate masks for all objects detected in the image\n",
    "sam2.generate(ortho_image_10cm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85",
   "metadata": {
    "id": "84"
   },
   "outputs": [],
   "source": [
    "# Save the generated masks as a GeoTIFF file\n",
    "# Each unique object gets a unique integer ID\n",
    "sam2.save_masks(output=\"masks.tif\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86",
   "metadata": {
    "id": "85"
   },
   "outputs": [],
   "source": [
    "sam2.show_masks(cmap=\"binary_r\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "87",
   "metadata": {
    "id": "86"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/fec9ad33-338e-4d1f-95e5-5f7b58e15edb)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "88",
   "metadata": {
    "id": "87"
   },
   "outputs": [],
   "source": [
    "sam2.show_masks(cmap=\"jet\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "89",
   "metadata": {
    "id": "88"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/32036cec-96a2-46c7-b948-a859925d8600)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90",
   "metadata": {
    "id": "89"
   },
   "source": [
    "Show the object annotations (objects with random color) on the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91",
   "metadata": {
    "id": "90"
   },
   "outputs": [],
   "source": [
    "sam2.show_anns(axis=\"off\", alpha=0.7, output=\"annotations.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92",
   "metadata": {
    "id": "91"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/8a05f797-5a08-4bb2-b6b3-ee8f82d8bd32)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "93",
   "metadata": {
    "id": "92"
   },
   "source": [
    "### Compare Original and Segmented Images\n",
    "\n",
    "Use an interactive slider to compare the original drone imagery with the SAM segmentation results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94",
   "metadata": {
    "id": "93"
   },
   "outputs": [],
   "source": [
    "leafmap.image_comparison(\n",
    "    ortho_image_10cm,\n",
    "    \"annotations.tif\",\n",
    "    label1=\"Drone Imagery\",\n",
    "    label2=\"Image Segmentation\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95",
   "metadata": {
    "id": "94"
   },
   "source": [
    "Add segmentation result to the map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96",
   "metadata": {
    "id": "95"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(ortho_image_10cm, layer_name=\"Ortho Imagery 202206\")\n",
    "m.add_raster(\"masks.tif\", colormap=\"jet\", layer_name=\"Masks\", nodata=0, opacity=0.7)\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97",
   "metadata": {
    "id": "96"
   },
   "source": [
    "Convert the object masks to vector format, such as GeoPackage, Shapefile, or GeoJSON."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "98",
   "metadata": {
    "id": "97"
   },
   "outputs": [],
   "source": [
    "sam2.raster_to_vector(\"masks.tif\", \"masks.gpkg\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99",
   "metadata": {
    "id": "98"
   },
   "outputs": [],
   "source": [
    "m.add_vector(\"masks.gpkg\", layer_name=\"Objects\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "100",
   "metadata": {
    "id": "99"
   },
   "source": [
    "### Automatic mask generation options\n",
    "\n",
    "There are several tunable parameters in automatic mask generation that control how densely points are sampled and what the thresholds are for removing low quality or duplicate masks. Additionally, generation can be automatically run on crops of the image to get improved performance on smaller objects, and post-processing can remove stray pixels and holes. Here is an example configuration that samples more masks:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "101",
   "metadata": {
    "id": "100"
   },
   "outputs": [],
   "source": [
    "sam2 = SamGeo2(\n",
    "    model_id=\"sam2-hiera-large\",\n",
    "    apply_postprocessing=False,\n",
    "    points_per_side=64,\n",
    "    points_per_batch=128,\n",
    "    pred_iou_thresh=0.7,\n",
    "    stability_score_thresh=0.92,\n",
    "    stability_score_offset=0.7,\n",
    "    crop_n_layers=1,\n",
    "    box_nms_thresh=0.7,\n",
    "    crop_n_points_downscale_factor=2,\n",
    "    min_mask_region_area=25,\n",
    "    use_m2m=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "102",
   "metadata": {
    "id": "101"
   },
   "outputs": [],
   "source": [
    "sam2.generate(ortho_image_10cm, output=\"masks2.tif\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "103",
   "metadata": {
    "id": "102"
   },
   "outputs": [],
   "source": [
    "sam2.show_masks(cmap=\"jet\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "104",
   "metadata": {
    "id": "103"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/ff07bff9-f0d1-41c6-a641-ce9f4734fd61)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "105",
   "metadata": {
    "id": "104"
   },
   "outputs": [],
   "source": [
    "sam2.show_anns(axis=\"off\", alpha=0.7, output=\"annotations2.tif\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "106",
   "metadata": {
    "id": "105"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/1e1df62c-ec44-4c5e-aa5a-4025d078ff6b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "107",
   "metadata": {
    "id": "106"
   },
   "outputs": [],
   "source": [
    "leafmap.image_comparison(\n",
    "    ortho_image_10cm,\n",
    "    \"annotations2.tif\",\n",
    "    label1=\"Image\",\n",
    "    label2=\"Image Segmentation\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "108",
   "metadata": {
    "id": "107"
   },
   "source": [
    "Remove small objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "109",
   "metadata": {
    "id": "108"
   },
   "outputs": [],
   "source": [
    "da, gdf = sam2.region_groups(\n",
    "    \"masks2.tif\",\n",
    "    connectivity=1,\n",
    "    min_size=10,\n",
    "    max_size=2000,\n",
    "    intensity_image=\"chm_202206.tif\",\n",
    "    out_image=\"objects.tif\",\n",
    "    out_csv=\"objects.csv\",\n",
    "    out_vector=\"objects.gpkg\",\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "110",
   "metadata": {
    "id": "kSqv_LxGLxgq"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(ortho_image_10cm, layer_name=\"Ortho Imagery 202206\")\n",
    "m.add_vector(\"objects.gpkg\", layer_name=\"Objects\")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "111",
   "metadata": {
    "id": "109"
   },
   "source": [
    "### Using box prompts\n",
    "\n",
    "Restart the Runtime to avoid the VRAM allocation issue."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "112",
   "metadata": {
    "id": "a3RypoEcNRM7"
   },
   "outputs": [],
   "source": [
    "import leafmap\n",
    "from samgeo import SamGeo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "113",
   "metadata": {
    "id": "0c4B40DENTYj"
   },
   "outputs": [],
   "source": [
    "ortho_image_10cm = \"ortho_image_202206_10cm.tif\"\n",
    "geojson = \"tree_boundaries.geojson\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "114",
   "metadata": {
    "id": "110"
   },
   "outputs": [],
   "source": [
    "gdf = leafmap.geojson_to_gdf(geojson)\n",
    "gdf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "115",
   "metadata": {
    "id": "112"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(ortho_image_10cm, layer_name=\"image\")\n",
    "style = {\n",
    "    \"color\": \"#ffff00\",\n",
    "    \"weight\": 2,\n",
    "    \"fillColor\": \"#7c4185\",\n",
    "    \"fillOpacity\": 0,\n",
    "}\n",
    "m.add_vector(geojson, style=style, zoom_to_layer=True, layer_name=\"Bounding boxes\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "116",
   "metadata": {
    "id": "113"
   },
   "outputs": [],
   "source": [
    "sam = SamGeo(\n",
    "    model_type=\"vit_h\",\n",
    "    automatic=False,\n",
    "    sam_kwargs=None,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "117",
   "metadata": {
    "id": "114"
   },
   "outputs": [],
   "source": [
    "sam.set_image(ortho_image_10cm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "118",
   "metadata": {
    "id": "115"
   },
   "outputs": [],
   "source": [
    "sam.predict(\n",
    "    boxes=geojson, point_crs=\"EPSG:4326\", output=\"tree_masks.tif\", dtype=\"uint16\"\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "119",
   "metadata": {
    "id": "116"
   },
   "outputs": [],
   "source": [
    "m.add_raster(\n",
    "    \"tree_masks.tif\", cmap=\"jet\", nodata=0, opacity=0.5, layer_name=\"Tree masks\"\n",
    ")\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "120",
   "metadata": {
    "id": "117"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/0a9b794d-cf94-4e5c-9134-8799e67d338c)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "121",
   "metadata": {
    "id": "118"
   },
   "source": [
    "## Training a Deep Learning Model for Tree Segmentation\n",
    "\n",
    "While SAM provides zero-shot segmentation, sometimes you need a custom model trained on your specific data. In this section, we'll train a U-Net model to segment tree canopies using labeled training data.\n",
    "\n",
    "### Why Train a Custom Model?\n",
    "\n",
    "- **Domain-specific accuracy**: Custom models learn features specific to your task (e.g., tree canopies vs. general objects)\n",
    "- **Consistency**: Produces consistent results across multiple images from the same sensor\n",
    "- **Speed**: Once trained, inference is faster than running SAM on large datasets\n",
    "- **Control**: You can tune the model to prioritize precision or recall based on your needs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "122",
   "metadata": {
    "id": "119"
   },
   "outputs": [],
   "source": [
    "import geoai\n",
    "import leafmap"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "123",
   "metadata": {
    "id": "120"
   },
   "source": [
    "### Prepare Training and Testing Datasets\n",
    "\n",
    "Deep learning models require labeled training data. We'll use tree boundary vectors from D2S to create training labels, then split our data into training and testing sets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "124",
   "metadata": {
    "id": "121"
   },
   "outputs": [],
   "source": [
    "m = leafmap.Map()\n",
    "m.add_raster(\"ortho_image_202206.tif\", layer_name=\"Ortho Imagery 202206\")\n",
    "m.add_vector(\"tree_boundaries.geojson\", layer_name=\"Tree Boundaries\")\n",
    "m"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "125",
   "metadata": {
    "id": "122"
   },
   "outputs": [],
   "source": [
    "# Define separate regions for training and testing\n",
    "# Training: Use June 2022 imagery with tree boundaries\n",
    "# Testing: Use December 2022 imagery to test model generalization\n",
    "bbox = [-97.956252, 26.165315, -97.954992, 26.165883]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "126",
   "metadata": {
    "id": "123"
   },
   "outputs": [],
   "source": [
    "ortho_image_202206 = \"ortho_image_202206.tif\"\n",
    "ortho_image_202212 = \"ortho_image_202212.tif\"\n",
    "train_raster_path = \"ortho_image_202206_train.tif\"\n",
    "test_raster_path = \"ortho_image_202212_test.tif\"\n",
    "train_vector_path = \"tree_boundaries.geojson\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "127",
   "metadata": {
    "id": "124"
   },
   "outputs": [],
   "source": [
    "# Clip training imagery to the training bbox\n",
    "# Resolution: 2cm (0.02m) provides good detail for tree segmentation\n",
    "train_image_array = leafmap.clip_raster(\n",
    "    ortho_image_202206,\n",
    "    geometry=bbox,\n",
    "    geom_crs=\"EPSG:4326\",\n",
    "    resolution=0.02,  # 2cm resolution\n",
    "    output=train_raster_path,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "128",
   "metadata": {
    "id": "125"
   },
   "outputs": [],
   "source": [
    "test_image_array = leafmap.clip_raster(\n",
    "    ortho_image_202212,\n",
    "    geometry=bbox,\n",
    "    geom_crs=\"EPSG:4326\",\n",
    "    resolution=0.02,\n",
    "    output=test_raster_path,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "129",
   "metadata": {
    "id": "126"
   },
   "source": [
    "### Generate Training Tiles\n",
    "\n",
    "Neural networks typically work with fixed-size inputs. We'll split the large training image into 512x512 pixel tiles with 256-pixel overlap to ensure complete coverage."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "130",
   "metadata": {
    "id": "127"
   },
   "outputs": [],
   "source": [
    "# Create training tiles from the imagery and tree boundaries\n",
    "# This will generate image/label pairs for U-Net training\n",
    "out_folder = \"trees\"\n",
    "tiles = geoai.export_geotiff_tiles(\n",
    "    in_raster=train_raster_path,  # Input imagery\n",
    "    out_folder=out_folder,  # Output directory\n",
    "    in_class_data=train_vector_path,  # Tree boundary labels\n",
    "    tile_size=512,  # Size of each tile in pixels\n",
    "    stride=256,  # Step between tiles (50% overlap)\n",
    "    buffer_radius=0,  # No buffer around geometries\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "131",
   "metadata": {
    "id": "128"
   },
   "source": [
    "### Train the U-Net Model\n",
    "\n",
    "Now we'll train a U-Net semantic segmentation model with a ResNet34 encoder (pretrained on ImageNet). This process includes:\n",
    "\n",
    "- **Architecture**: U-Net with ResNet34 backbone for feature extraction\n",
    "- **Training data**: 288 tiles split into 80% train / 20% validation\n",
    "- **Optimization**: Adam optimizer with learning rate 0.001\n",
    "- **Epochs**: 30 training epochs with best model checkpointing\n",
    "- **Metrics**: IoU (Intersection over Union), F1-score, Precision, and Recall\n",
    "\n",
    "Training on a GPU (T4 or better) typically takes 5-10 minutes for this dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "132",
   "metadata": {
    "id": "129"
   },
   "outputs": [],
   "source": [
    "# Train U-Net model\n",
    "geoai.train_segmentation_model(\n",
    "    images_dir=f\"{out_folder}/images\",\n",
    "    labels_dir=f\"{out_folder}/labels\",\n",
    "    output_dir=f\"{out_folder}/unet_models\",\n",
    "    architecture=\"unet\",\n",
    "    encoder_name=\"resnet34\",\n",
    "    encoder_weights=\"imagenet\",\n",
    "    num_channels=4,\n",
    "    num_classes=2,  # background and trees\n",
    "    batch_size=8,\n",
    "    num_epochs=10,\n",
    "    learning_rate=0.001,\n",
    "    val_split=0.2,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "133",
   "metadata": {
    "id": "130"
   },
   "outputs": [],
   "source": [
    "geoai.plot_performance_metrics(\n",
    "    history_path=f\"{out_folder}/unet_models/training_history.pth\",\n",
    "    figsize=(15, 5),\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "134",
   "metadata": {
    "id": "131"
   },
   "source": [
    "![metrics](https://github.com/user-attachments/assets/17f13951-1805-43c9-8eac-0c74acd303cc)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "135",
   "metadata": {
    "id": "132"
   },
   "source": [
    "```text\n",
    "=== Performance Metrics Summary ===\n",
    "IoU     - Best: 0.9454 | Final: 0.9420\n",
    "F1      - Best: 0.9718 | Final: 0.9700\n",
    "Precision - Best: 0.9715 | Final: 0.9700\n",
    "Recall  - Best: 0.9722 | Final: 0.9702\n",
    "Val Loss - Best: 0.0746 | Final: 0.0829\n",
    "===================================\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "136",
   "metadata": {
    "id": "133"
   },
   "outputs": [],
   "source": [
    "model_path = f\"{out_folder}/unet_models/best_model.pth\"\n",
    "masks_path = \"ortho_image_202212_masks.tif\"\n",
    "probability_path = \"probabilities.tif\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "137",
   "metadata": {
    "id": "134"
   },
   "source": [
    "### Apply Model to Test Imagery\n",
    "\n",
    "Use the trained model to predict tree canopies on the December 2022 test imagery. We'll save both:\n",
    "- **Classification mask**: Final tree/background predictions\n",
    "- **Probability map**: Per-pixel confidence scores for each class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "138",
   "metadata": {
    "id": "135"
   },
   "outputs": [],
   "source": [
    "# Run inference on the test image using the trained model\n",
    "geoai.semantic_segmentation(\n",
    "    input_path=test_raster_path,  # Input test imagery\n",
    "    output_path=masks_path,  # Output classification mask\n",
    "    model_path=model_path,  # Path to trained model weights\n",
    "    architecture=\"unet\",  # Same architecture as training\n",
    "    encoder_name=\"resnet34\",  # Same encoder as training\n",
    "    num_channels=4,  # RGB + NIR bands\n",
    "    num_classes=2,  # Background (0) and trees (1)\n",
    "    window_size=512,  # Process in 512x512 windows\n",
    "    overlap=256,  # 50% overlap for smooth boundaries\n",
    "    batch_size=8,  # Process 8 windows at a time\n",
    "    probability_threshold=0.5,  # Confidence threshold for classification\n",
    "    probability_path=probability_path,  # Save probability scores\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "139",
   "metadata": {
    "id": "136"
   },
   "source": [
    "### Visualize Results\n",
    "\n",
    "Display the test imagery, predicted segmentation masks, and probability maps on an interactive map. The probability layer shows the model's confidence for each pixel (higher values = more confident)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "140",
   "metadata": {
    "id": "137"
   },
   "outputs": [],
   "source": [
    "# Create an interactive map to visualize the results\n",
    "m = leafmap.Map()\n",
    "\n",
    "# Add the original test imagery as the base layer\n",
    "m.add_raster(test_raster_path, layer_name=\"Ortho Imagery 202212\")\n",
    "\n",
    "# Add the segmentation mask (hidden by default)\n",
    "m.add_raster(\n",
    "    masks_path,\n",
    "    colormap=\"jet\",\n",
    "    layer_name=\"Segmentation\",\n",
    "    nodata=0,\n",
    "    visible=False,  # Hidden by default, can toggle on\n",
    ")\n",
    "\n",
    "# Add the probability map showing model confidence\n",
    "# Band 2 contains the probability for class 1 (trees)\n",
    "m.add_raster(\n",
    "    probability_path,\n",
    "    indexes=[2],  # Class 1 (tree) probability\n",
    "    colormap=\"jet\",  # Color scale: blue (low) to red (high confidence)\n",
    "    opacity=0.5,  # Semi-transparent overlay\n",
    "    layer_name=\"Tree Probability\",\n",
    "    nodata=0,\n",
    ")\n",
    "\n",
    "m"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "141",
   "metadata": {
    "id": "138"
   },
   "source": [
    "![image](https://github.com/user-attachments/assets/8bd2af2b-dd13-48c1-a4f1-c6bbe0faff00)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "142",
   "metadata": {
    "id": "139"
   },
   "source": [
    "## Summary and Next Steps\n",
    "\n",
    "In this workshop, you learned how to:\n",
    "\n",
    "1. **Access and visualize drone imagery** from the Data to Science (D2S) platform\n",
    "2. **Perform zero-shot segmentation** using SAM2 for automatic object detection\n",
    "3. **Train custom deep learning models** (U-Net) for domain-specific segmentation tasks\n",
    "4. **Apply trained models** to new imagery and evaluate results\n",
    "\n",
    "### Key Takeaways\n",
    "\n",
    "- **SAM2** is excellent for exploratory analysis and when you don't have training labels\n",
    "- **Custom U-Net models** provide better accuracy for specific tasks when training data is available\n",
    "- The **GeoAI package** simplifies the entire pipeline from data preparation to model deployment\n",
    "- **Leafmap** makes it easy to visualize and interact with geospatial AI results\n",
    "\n",
    "### Next Steps\n",
    "\n",
    "- Apply these techniques to your own drone or satellite imagery\n",
    "- Experiment with different model architectures (DeepLabV3+, FPN, etc.)\n",
    "- Try transfer learning from pre-trained models on similar agricultural datasets\n",
    "- Explore temporal analysis by comparing multiple flights over time\n",
    "- Integrate with other geospatial analysis workflows (crop health, yield prediction, etc.)\n",
    "\n",
    "### Resources\n",
    "\n",
    "- **GeoAI Documentation**: https://opengeoai.org\n",
    "- **Leafmap Documentation**: https://leafmap.org\n",
    "- **SAMGeo Documentation**: https://samgeo.gishub.org\n",
    "- **D2S website**: https://d2s.org\n",
    "- **D2S Platform**: https://ps2.d2s.org\n",
    "- **D2S STAC**: https://stac.d2s.org \n",
    "- **D2S GitHub**: https://github.com/gdslab/data-to-science\n",
    "\n",
    "### Questions?\n",
    "\n",
    "Feel free to reach out to the GeoAI community on GitHub."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "gpuType": "T4",
   "provenance": []
  },
  "jupytext": {
   "default_lexer": "ipython3"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
